perf(cli): optimize quality scoring and deterministic merge allocations by d-oit · Pull Request #640 · d-oit/do-web-doc-resolver

d-oit · 2026-09-25T11:18:27Z

What

Two CLI files, both genuine wins:

cli/src/synthesis.rs: hoists the link regex into a LazyLock. main
recompiled https?://... on every deterministic_merge() call.
cli/src/synthesis.rs: seen_lines.insert(trimmed) instead of
insert(trimmed.to_string()), removing a heap allocation per content line,
plus HashSet::with_capacity.
cli/src/quality.rs: has_frontmatter now inspects only the frontmatter
block, so field names appearing in the body after the closing delimiter can
no longer spoof the bonus.
cli/src/quality.rs: keeps INITIAL_LINE_CAPACITY, and derives the header
end from the delimiter lengths instead of a magic +7.
cli/tests/quality.rs: regression test proving the spoofed case no longer
earns the bonus.

Rebased

The original branch also carried 11 files that had already landed on main:
nine dtolnay/rust-toolchain SHA bumps and four web/app/* accessibility
edits from #639. main is merged in and those are dropped, so the PR is now
exactly the two CLI files plus the test.

- Pre-calculate total lines using byte scanning and early exit duplicate checking once unique line threshold is met in `score_content`. - Restrict YAML frontmatter checks to header slice and guard structural anchor searches in `score_content`. - Pre-allocate HashSet capacity (128) and use `&str` instead of `String` allocations for line deduplication in `deterministic_merge`. - Reuse static `LINK_REGEX` in `deterministic_merge` to avoid per-call regex compilation.

vercel · 2026-09-25T11:18:27Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
do-web-doc-resolover	Ready	Preview	Oct 1, 2026 5:06pm UTC

google-labs-jules · 2026-09-25T11:18:28Z

👋 Jules, reporting for duty! I'm here to lend a hand with this pull request.

When you start a review, I'll add a 👀 emoji to each comment to let you know I've read it. I'll focus on feedback directed at me and will do my best to stay out of conversations between you and other bots or reviewers to keep the noise down.

I'll push a commit with your requested changes shortly after. Please note there might be a delay between these steps, but rest assured I'm on the job!

For more direct control, you can switch me to Reactive Mode. When this mode is on, I will only act on comments where you specifically mention me with @jules. You can find this option in the Pull Request section of your global Jules UI settings. You can always switch back!

New to Jules? Learn more at jules.google/docs.

For security, I will only act on instructions from the user who triggered this task.

codacy-production · 2026-09-25T11:21:57Z

Up to standards ✅

🟢 Issues 0 issues

Results:
0 new issues

View in Codacy

🟢 Metrics 0 complexity · 0 duplication

Metric Results

Complexity 0

Duplication 0

View in Codacy

_{NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer}
_{TIP This summary will be updated as you push new changes.}

- Restrict YAML frontmatter checks to header slice in score_content. - Pre-allocate HashSet capacity and use &str for line deduplication. - Reuse static LINK_REGEX in deterministic_merge to avoid per-call compile.

cline-cloud · 2026-10-01T16:12:46Z

🔥 Roast: a `perf(cli)` PR where 10 of 13 files are already in `main`

I diffed this branch's head tree against current main rather than against its own base. The effective unique content is two files — cli/src/quality.rs and cli/src/synthesis.rs.

The other 11 files are déjà vu

9 × dtolnay/rust-toolchain SHA bumps to 02cb101e…. main already runs 02cb101ec7c40f2c49e1d9714d64511d8e1b74de in all 9 workflows.
4 × web/app/* accessibility edits — the focus-visible:outline-* rings and aria-label="Raw markdown result". Already on main via merged feat(ux): add missing keyboard focus indicators and aria labels #639, byte for byte.

That is 77% of the diff re-proposing work that has already landed, which is also why this PR reports CONFLICTING/DIRTY. A perf(cli) PR whose largest component is a re-application of someone else's merged accessibility patch is not a perf(cli) PR.

The 23% that is new is genuinely good

LINK_REGEX as a LazyLock — this is the real find. main compiles the link regex on every single call to deterministic_merge():
```
// synthesis.rs:289, on main
let link_re = regex::Regex::new(r"https?://[^\s)>\]]+").unwrap();
```
Hoisting that into a LazyLock is a correct and worthwhile fix.
seen_lines.insert(trimmed) instead of insert(trimmed.to_string()) — drops one heap allocation per content line. Lifetimes check out (results: &[ResolvedResult] outlives the set).
HashSet::with_capacity(128) in deterministic_merge.
Scoping has_frontmatter to the actual frontmatter block instead of running contains() over the entire document. Strictly stricter, and a genuine correctness improvement — a body line mentioning last_updated: can no longer fake a frontmatter bonus.

But it also walks one thing backwards

-const INITIAL_LINE_CAPACITY: usize = 128;
-
-    let mut unique_set = std::collections::HashSet::with_capacity(INITIAL_LINE_CAPACITY);
+    let mut unique_set = std::collections::HashSet::with_capacity(128);

That constant is in main because of the earlier quality-scoring optimization work. A perf(cli) PR that reintroduces the magic number it was extracted from is moving in the wrong direction. Please keep the named constant — the PR's own stated theme is allocation hygiene.

Nits

rest.find("\n---").map(|i| i + 7) — the 7 is an undocumented magic number that only works because "---" is 3 bytes and "\n---" is 4. Change either and the slice is silently wrong. Derive it from the lengths.
If there is no closing ---, unwrap_or(trimmed.len()) scans the whole document — silently reverting to the old loose behaviour. Intentional? Worth a comment either way.
commitlint is red on body-max-line-length: the Jules boilerplate footer URL exceeds 100 chars. Same failure as every other Jules PR here.

Verdict

Keep — the CLI changes are worth landing and are not on main yet.

But it needs work before it is mergeable: rebase onto main, drop all 11 already-merged files, restore INITIAL_LINE_CAPACITY, and retitle it to describe the two files it actually changes. As it stands the only reviewable content is buried under a re-run of history.

# Conflicts: # .github/workflows/yaml-lint.yml

Restores INITIAL_LINE_CAPACITY rather than inlining the literal 128, and derives the frontmatter header end from the delimiter lengths instead of the magic offset +7. Adds a regression test proving that field names appearing only in the body, after the closing delimiter, no longer earn the frontmatter bonus. Refs #640

The regression test asserted a 0.05 gap on content that already scored 1.0, where the final score.clamp(0.0, 1.0) hides the bonus entirely. Passing empty links applies the missing-links penalty, leaving headroom (real 0.95 vs spoofed 0.90). Also drops a stray blank line and a trailing blank line that cargo fmt --check rejected. Refs #640

- Restrict YAML frontmatter checks to header slice in score_content. - Pre-allocate HashSet capacity and use &str for line deduplication. - Reuse static LINK_REGEX in deterministic_merge to avoid per-call compile.

…-opt' (keep reviewed tree)

cargo fmt --check failed on cli/tests/quality.rs:42. Scanned the touched Rust files and collapsed the remaining consecutive blank lines. Refs #640

vercel Bot deployed to Preview September 25, 2026 11:18 View deployment

perf(cli): optimize quality scoring and synthesis merge

d0e59a6

- Restrict YAML frontmatter checks to header slice in score_content. - Pre-allocate HashSet capacity and use &str for line deduplication. - Reuse static LINK_REGEX in deterministic_merge to avoid per-call compile.

vercel Bot deployed to Preview September 25, 2026 11:32 View deployment

perf(cli): optimize quality scoring and synthesis merge

a0c8d3e

- Restrict YAML frontmatter checks to header slice in score_content. - Pre-allocate HashSet capacity and use &str for line deduplication. - Reuse static LINK_REGEX in deterministic_merge to avoid per-call compile.

vercel Bot deployed to Preview September 25, 2026 11:39 View deployment

perf(cli): optimize quality scoring and synthesis merge

d6a9570

- Restrict YAML frontmatter checks to header slice in score_content. - Pre-allocate HashSet capacity and use &str for line deduplication. - Reuse static LINK_REGEX in deterministic_merge to avoid per-call compile.

vercel Bot deployed to Preview September 25, 2026 11:42 View deployment

d-oit added 2 commits October 1, 2026 16:19

Merge remote-tracking branch 'origin/main' into pr640

7ca3283

# Conflicts: # .github/workflows/yaml-lint.yml

vercel Bot deployed to Preview October 1, 2026 16:21 View deployment

vercel Bot deployed to Preview October 1, 2026 16:32 View deployment

perf(cli): optimize quality scoring and synthesis merge

a2da274

- Restrict YAML frontmatter checks to header slice in score_content. - Pre-allocate HashSet capacity and use &str for line deduplication. - Reuse static LINK_REGEX in deterministic_merge to avoid per-call compile.

vercel Bot deployed to Preview October 1, 2026 16:36 View deployment

perf(cli): optimize quality scoring and synthesis merge

1e561ff

- Restrict YAML frontmatter checks to header slice in score_content. - Pre-allocate HashSet capacity and use &str for line deduplication. - Reuse static LINK_REGEX in deterministic_merge to avoid per-call compile.

vercel Bot deployed to Preview October 1, 2026 16:52 View deployment

d-oit added 2 commits October 1, 2026 16:58

Merge remote-tracking branch 'origin/main' into pr640

c3bcd8e

Merge remote-tracking branch 'origin/perf/cli-quality-synthesis-alloc…

4a9396b

…-opt' (keep reviewed tree)

vercel Bot deployed to Preview October 1, 2026 17:00 View deployment

style(cli): collapse blank line rejected by cargo fmt

0d8a84a

cargo fmt --check failed on cli/tests/quality.rs:42. Scanned the touched Rust files and collapsed the remaining consecutive blank lines. Refs #640

vercel Bot deployed to Preview October 1, 2026 17:06 View deployment

cline-cloud Bot merged commit 20ed4d4 into main Oct 1, 2026
46 of 47 checks passed

cline-cloud Bot deleted the perf/cli-quality-synthesis-alloc-opt-8537655933945029810 branch October 1, 2026 17:13

Navigation Menu

perf(cli): optimize quality scoring and deterministic merge allocations - #640

perf(cli): optimize quality scoring and deterministic merge allocations#640
cline-cloud[bot] merged 12 commits into
mainfrom
perf/cli-quality-synthesis-alloc-opt-8537655933945029810

d-oit commented Sep 25, 2026 •

edited by cline-cloud Bot

Loading

Uh oh!

vercel Bot commented Sep 25, 2026 •

edited

Loading

Uh oh!

google-labs-jules Bot commented Sep 25, 2026

Uh oh!

codacy-production Bot commented Sep 25, 2026

Uh oh!

cline-cloud Bot commented Oct 1, 2026

Uh oh!

Uh oh!

This branch was successfully deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

d-oit commented Sep 25, 2026 • edited by cline-cloud Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What

Rebased

Uh oh!

vercel Bot commented Sep 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

google-labs-jules Bot commented Sep 25, 2026

Uh oh!

codacy-production Bot commented Sep 25, 2026

Up to standards ✅

Uh oh!

cline-cloud Bot commented Oct 1, 2026

🔥 Roast: a perf(cli) PR where 10 of 13 files are already in main

The other 11 files are déjà vu

The 23% that is new is genuinely good

But it also walks one thing backwards

Nits

Verdict

Uh oh!

Uh oh!

This branch was successfully deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

d-oit commented Sep 25, 2026 •

edited by cline-cloud Bot

Loading

vercel Bot commented Sep 25, 2026 •

edited

Loading

🔥 Roast: a `perf(cli)` PR where 10 of 13 files are already in `main`